The number of international benchmarking competitions is steadily increasing in various fields of machine learning (ML) research and practice. So far, however, little is known about the common practice as well as bottlenecks faced by the community in tackling the research questions posed. To shed light on the status quo of algorithm development in the specific field of biomedical imaging analysis, we designed an international survey that was issued to all participants of challenges conducted in conjunction with the IEEE ISBI 2021 and MICCAI 2021 conferences (80 competitions in total). The survey covered participants' expertise and working environments, their chosen strategies, as well as algorithm characteristics. A median of 72% challenge participants took part in the survey. According to our results, knowledge exchange was the primary incentive (70%) for participation, while the reception of prize money played only a minor role (16%). While a median of 80 working hours was spent on method development, a large portion of participants stated that they did not have enough time for method development (32%). 25% perceived the infrastructure to be a bottleneck. Overall, 94% of all solutions were deep learning-based. Of these, 84% were based on standard architectures. 43% of the respondents reported that the data samples (e.g., images) were too large to be processed at once. This was most commonly addressed by patch-based training (69%), downsampling (37%), and solving 3D analysis tasks as a series of 2D tasks. K-fold cross-validation on the training set was performed by only 37% of the participants and only 50% of the participants performed ensembling based on multiple identical models (61%) or heterogeneous models (39%). 48% of the respondents applied postprocessing steps.

The real-world data tends to be heavily imbalanced and severely skew the data-driven deep neural networks, which makes Long-Tailed Recognition (LTR) a massive challenging task. Existing LTR methods seldom train Vision Transformers (ViTs) with Long-Tailed (LT) data, while the off-the-shelf pretrain weight of ViTs always leads to unfair comparisons. In this paper, we systematically investigate the ViTs' performance in LTR and propose LiVT to train ViTs from scratch only with LT data. With the observation that ViTs suffer more severe LTR problems, we conduct Masked Generative Pretraining (MGP) to learn generalized features. With ample and solid evidence, we show that MGP is more robust than supervised manners. In addition, Binary Cross Entropy (BCE) loss, which shows conspicuous performance with ViTs, encounters predicaments in LTR. We further propose the balanced BCE to ameliorate it with strong theoretical groundings. Specially, we derive the unbiased extension of Sigmoid and compensate extra logit margins to deploy it. Our Bal-BCE contributes to the quick convergence of ViTs in just a few epochs. Extensive experiments demonstrate that with MGP and Bal-BCE, LiVT successfully trains ViTs well without any additional data and outperforms comparable state-of-the-art methods significantly, e.g., our ViT-B achieves 81.0% Top-1 accuracy in iNaturalist 2018 without bells and whistles. Code is available at https://github.com/XuZhengzhuo/LiVT.

Image super-resolution is a common task on mobile and IoT devices, where one often needs to upscale and enhance low-resolution images and video frames. While numerous solutions have been proposed for this problem in the past, they are usually not compatible with low-power mobile NPUs having many computational and memory constraints. In this Mobile AI challenge, we address this problem and propose the participants to design an efficient quantized image super-resolution solution that can demonstrate a real-time performance on mobile NPUs. The participants were provided with the DIV2K dataset and trained INT8 models to do a high-quality 3X image upscaling. The runtime of all models was evaluated on the Synaptics VS680 Smart Home board with a dedicated edge NPU capable of accelerating quantized neural networks. All proposed solutions are fully compatible with the above NPU, demonstrating an up to 60 FPS rate when reconstructing Full HD resolution images. A detailed description of all models developed in the challenge is provided in this paper.

Various depth estimation models are now widely used on many mobile and IoT devices for image segmentation, bokeh effect rendering, object tracking and many other mobile tasks. Thus, it is very crucial to have efficient and accurate depth estimation models that can run fast on low-power mobile chipsets. In this Mobile AI challenge, the target was to develop deep learning-based single image depth estimation solutions that can show a real-time performance on IoT platforms and smartphones. For this, the participants used a large-scale RGB-to-depth dataset that was collected with the ZED stereo camera capable to generated depth maps for objects located at up to 50 meters. The runtime of all models was evaluated on the Raspberry Pi 4 platform, where the developed solutions were able to generate VGA resolution depth maps at up to 27 FPS while achieving high fidelity results. All models developed in the challenge are also compatible with any Android or Linux-based mobile devices, their detailed description is provided in this paper.

Dense pose estimation is a dense 3D prediction task for instance-level human analysis, aiming to map human pixels from an RGB image to a 3D surface of the human body. Due to a large amount of surface point regression, the training process appears to be easy to collapse compared to other region-based human instance analyzing tasks. By analyzing the loss formulation of the existing dense pose estimation model, we introduce a novel point regression loss function, named Dense Points} loss to stable the training progress, and a new balanced loss weighting strategy to handle the multi-task losses. With the above novelties, we propose a brand new architecture, named UV R-CNN. Without auxiliary supervision and external knowledge from other tasks, UV R-CNN can handle many complicated issues in dense pose model training progress, achieving 65.0% $AP_{gps}$ and 66.1% $AP_{gpsm}$ on the DensePose-COCO validation subset with ResNet-50-FPN feature extractor, competitive among the state-of-the-art dense human pose estimation methods.

基于匹配的方法，尤其是基于时空记忆的方法，在半监督视频对象分割（VOS）中明显领先于其他解决方案。但是，不断增长和冗余的模板特征导致推断效率低下。为了减轻这一点，我们提出了一个新型的顺序加权期望最大化（SWEM）网络，以大大降低记忆特征的冗余。与以前仅检测帧之间特征冗余的方法不同，Swem通过利用顺序加权EM算法来合并框架内和框架间的相似特征。此外，框架特征的自适应权重具有代表硬样品的灵活性，从而改善了模板的歧视。此外，该提出的方法在内存中保留了固定数量的模板特征，从而确保了VOS系统的稳定推理复杂性。对常用的戴维斯和YouTube-VOS数据集进行了广泛的实验，验证了SWEM的高效率（36 fps）和高性能（84.3 \％$ \ Mathcal {J} \＆\ Mathcal {F} $代码可在以下网址获得：https：//github.com/lmm077/swem。

人类垫子是指从具有高质量的自然图像中提取人类部位，包括人类细节信息，例如头发，眼镜，帽子等。这项技术在电影行业的图像合成和视觉效果中起着至关重要的作用。当绿屏不可用时，现有的人类底漆方法需要其他输入（例如Trimap，背景图像等）或具有较高计算成本和复杂网络结构的模型，这给应用程序带来了很大的困难实践中的人类垫子。为了减轻此类问题，大多数现有方法（例如MODNET）使用多分支为通过细分铺平道路，但是这些方法并未充分利用图像功能，并且仅利用网络的预测结果作为指导信息。因此，我们提出了一个模块来生成前景概率图，并将其添加到MODNET中以获得语义引导的Matting Net（SGM-NET）。在只有一个图像的条件下，我们可以实现人类的效果任务。我们在P3M-10K数据集上验证我们的方法。与基准相比，在各种评估指标中，我们的方法显着改善。

最近，机器学习（ML）电位的发展使得以量子力学（QM）模型的精度进行大规模和长期分子模拟成为可能。但是，对于高水平的QM方法，例如在元gga级和/或具有精确交换的密度函数理论（DFT），量子蒙特卡洛等，生成足够数量的用于训练的数据由于其高成本，计算挑战性。在这项工作中，我们证明了基于ML的DFT模型Deep Kohn-Sham（Deepks）可以在很大程度上缓解这个问题。 DeepKS采用计算高效的基于神经网络的功能模型来构建在廉价DFT模型上添加的校正项。在训练后，DeepKs提供了与高级QM方法相比，具有紧密匹配的能量和力，但是所需的训练数据的数量是比训练可靠的ML潜力所需的数量级要小。因此，DeepKs可以用作昂贵的QM型号和ML电位之间的桥梁：一个人可以生成相当数量的高准确性QM数据来训练DeepKs模型，然后使用DeepKs型号来标记大量的配置以标记训练ML潜力。该周期系统方案在DFT软件包算盘中实施，该计划是开源的，可以在各种应用程序中使用。

场景文本识别是一个流行的主题，在行业中广泛使用。尽管许多方法在封闭式文本识别挑战方面取得了令人满意的性能，但这些方法在开放式场景中丧失了可行性，在开放式场景中，收集数据或新颖性格的重新培训可能会产生高成本。例如，对外语的注释样本可能很昂贵，而每次从历史文档中发现新颖角色时，请重新训练该模型。在本文中，我们介绍并制定了一项新的开放式文本识别任务，该任务要求能够发现和识别新颖的角色而无需再培训。标签到原型学习框架也被提议作为建议任务的基准。具体而言，该框架引入了可推广的标签到原型映射功能，以构建可见和看不见类的原型（类中心）。然后使用开放式预测指标来识别或拒绝样品。在集合字符上的拒绝能力实现允许在传入数据流中自动发现未知字符。广泛的实验表明，我们的方法在各种零射击，封闭设置和开放式文本识别数据集上实现了有希望的性能

对抗性培训（AT）被认为是对抗对抗攻击最可靠的防御之一。然而，模型培训以牺牲标准精度，并不概括为新的攻击。最近的作用表明，在新型威胁模型中的新威胁模型或神经感知威胁模型中，对普遍威胁模型的对抗样本进行了泛化改进。然而，前者需要确切的流形信息，而后者需要算法放松。通过这些考虑因素，我们利用了具有规范化流的底层歧管信息，确保了确切的歧管的假设保持。此外，我们提出了一种名为联合空间威胁模型（JSTM）的新型威胁模型，其可以作为神经感知威胁模型的特殊情况，这些威胁模型不需要额外放松来制作相应的对抗性攻击。在JSTM下，我们培养了新的对抗性攻击和防御。混合策略提高了神经网络的标准准确性，但与AT结合时牺牲了鲁棒性。为了解决这个问题，我们提出了强大的混合策略，其中我们最大限度地提高了内插图像的逆境，并获得了鲁棒性和预装配。我们的实验表明，内插关节空间对抗性训练（IJSAT）在CiFar-10/100，Om-ImageNet和CiFar-10-C数据集中实现了标准精度，鲁棒性和泛化的良好性能。 IJSAT也是灵活的，可以用作数据增强方法，以提高标准精度，并与诸多换取以提高鲁棒性的方法相结合。